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ABSTRACT 

A study assessed the relative effectiveness of four 
types of follow-up techniques in increasing responses to mailed 
questionnaires requiring responses from multiple individuals. The 
respondents were college administrators. Copies of questionnaires 
designed to measure professional development were mailed to a random 
sample of 693 chief student affairs officers who were members of the 
National Association of School Personnel Administrators. The subjects 
were asked to complete and return the measure and forward a copy to 
their supervisor and to the student government president tor 
completion and return. A total of 91 subjects responded to the 
initial mailing, and 113 subjects indicated an inability or 
unwillingness to participate in the study, leaving 344 subjects 
available for the study on follow-up approaches. Follow-up methods 
chosen included: (1) a postcard requesting a response; (2) an 
individually addressed letter ^yped on letterhead; (3) a photocopied 
"Dear Colleague" letter; and (4) telephone calls. In addition to 
requesting a response, each method included an offer to forward 
another set of survey materials. Results indicate that the degree of 
personalization and reduction of costs for the subjects' responses 
were the most salient factors in each method's success. (TJH) 
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Pour Methoas of Followir^-up 
Mailed Qi^tionnaires 



Mail questionnaire has iseen one o£ tl^ most popular methods of 
survey data collection because of its low cost and relative ease of 
adainistraticm. The major drai*ack that it presents is tte sosaetisaes 
lov rate of response. Vhls is especially true in situations requiring 
multiple responses fro® each observational unit. The current study vas 
designed to assess the relative effectiveness of four different types of 
follov-up techniques in increasing responses in just such a situaticHi. 

Goy:3er (1985) reports that the average 7.5% advantage in response 
rate of interviews over mail surveys hajs all but disappeared over time. 
He attributes much of the historical difference to more aggressive 
follow-up in interviews and suggests that low response rate in mail 
surveys can be meliorated especially throi^h follow-up. In a review of 
literature, Linsky (1975) indicated factors that have been shown to have 
an inject upon response rates • These include mechanical and perceptual 
factors, broad motivational factors, and mor^tary revards and other 
motivational factors. 

Mechanical and perceptual factors include: precontact by letter, 
postcard or telefdione; enclosure of a postcard to return to signify 
cosipletion of an anonymous response; follow up; hig^r powered postage 
on outgoing arid return envelopes; length of the qi^tionnaire; printing 
of the questionnaires (color, size, and so on), and preceding of the 
questionnaire. 

tkodd motivational factors include: anonymity; cover letters; 
personalization; place and In^rtance of the respondent; argun^nt for 
social utility; a^^eal to help those conducting tte study; 
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identification of sponsoring organization and titles o£ researchers; and 
thB use of deadlines. 

Direct motivational facuors include: cash regards; and the 
enclosure of prizes, premiuios, or other non-<:ash rewards. 

Personalization is one of the techniques nost coiamonly believed to 
produce an increase in response. A number of studies have found this 
effect. Hochstim and Athenosopoulos (1970) and Shackelton and Wild 
(1982) both found positive effects for persoral contact between the 
researcher and tte subject. In the forxeer study costs were sli^^tly 
more than double that of ceil contact, and in the latter the eiEfect was 
found only in interacticm with a financial incentive. However, House, 
Gerber, and McMichael (1977) found use of certified mail to be superior 
- to hand delivery of a follow-up by an enployee's supervisor in a study 
of canufacturing plant workers. 

Another aspect of personalization is a aore personalized approach 
in the material mailed to the potential respondee* Linsky (1965) and 
Dillman and Frey (1974) found a positive effect for individually 
addressed letters. Martin, Duncan and Sawyer (1984) found significant 
effects of personalization only in interaction with prenotification. 
Snelling (1969 ) reported achieving a 92 .62% response rate among 
graduates of four liberal arts colleges using a highly personalized 
approach. This included initial letters individually typed, 
incorporating nicknames or other individually meaningful references, 
and signed by a professor with whom the student had shared rapport. 
This was followed by an autcxnatic reminder from th^ project director, 
and then an individually typed letter signed by the respective collet 
president if no response yfa& received. Anderson and Berdie (1975) got 
mixed results, with personalization techniques such as humorous post 
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cards and hand addressing. These were found successful with college 
students, Imt not with professors or administrators* 

Other researchers have found no effect of personalization including 
Lahrecque (1977), Worthen and Valcarce (1985), Woodverd and MdKelvie 
(1985), and Haves, Crittenden and Crittenden, (1987). S\;ill others such 
as Andreason (1970) h?ve fouaid lower response rates 'Anon personalization 
vas used. These conflicting results have pr<^ted authors such as Trice 
(1985) and Heberlein and BauiDc^rtner (1978) to suggest influences such 
as conflict with the effects of anonyaaity and a connection with the 
perceived iioportance of the individual to the study as e3q>lanations. 

Heberlein and Baungartner (1978) identify perceived ii^rtance and 
decreased response cost ( to the respondee ) as factors affecting 
response. Linsky (1965) fouxvl iii^rtance of response to be an in^rtant 
factor in response rate, axyi Tedin and Hofstetter (1982) found tte 
importance factor to be more salient than cost factors as an influence 
upon response rates. Hhey found a certified letter effective in 
establishing importance. 

Use of certified nail creates a sense of dissonance or guilt, so 
respondents attempt to reduce that by returning the questionnaires. 
Telephone calls are ^r^rally considered to be most effective. 

Among those researchers studying the effects of telephone calls, 
RoBcoe, Lang, ar)d Sheth (1975) concluded that the tele^^ne reminder was 
consistently more effective. Heberlein and Baumgartner (1978) in their 
Gietaanal^^is found telephor^ calls to be effective, especially when used 
on secoiKj or third contacts. . Finally, Dlllman and Frey (1974) found no 
difference in response rates using telephone contacts, but did find an 
increase In response speed. 
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Affiong the iK)3t effective are the mechanical and perceptual factors 
including follow-up. Follow vxp actions can include letters;^ postcards, 
telefAione calls, or certified mail ^ich are \2sed to encourage the 
respondents to return the questionnaireis. In his review, Ifervey (1987) 
cited use of follow-i^ as an essential in increasing response rates. 
Iteberlein and Baunigartner (1978) in an extensive metaanalysis indicated 
that the optimum nusber of follow-up contacts generally is three. Hhey 
further went cm to caution that through their regression analysis of 
response rate research, only 40% of the variance could be explained by 
the factors considered. lliis is consistent with an interpretation of a 
coQ^lex, interactive influence of variables upon response rate. 

Miller and Smith {1983) fouiKl that a second Hailing of the coK¥)lete 
packet produced 70%-90% return rate increases. Putrell and Lamb (1981) 
found that both the number of follow-up contacts, ard including a 
quest lonraire with the follow-up irinxeased response rates, with an 
overall response rate of 34.% with, questionnaire catjpared to 20.3% 
without. Keberlein arK3 Baumgartner (1931) in a coo^xDSite of 13 studies 
found sli^t positive effects for the inclusion of a second 
questionnaire with a follow-up letter. 'n>ey report that approximately 
90% of the effect can be obtained with the follow-up letter alone, 
suggesting that the 10% incremental increase be weighed against the 
added cost. 

Linsky (1975) reports 58% response by using letters and 53% with 
postcards. Von Reisen (1979) found that a follow-t?) consisting of a 
letter and a questionnaire was superior to a postcard follow-up or no 
follow-up, tHit found no significant difference between a postcard and no 
follow-up in a study of veterinarians. 
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\ follov-up technique combined with incentives increases the rate 
of return. Pucel, et al • ( 1968 ) , cooobined follow-up method with 
different kinds of incentives, and found no dif fereiKres among four 
different incentives (pencil, colored questionnaire, packet of instant 
coffee, or preletter), but that the rate of response is related to the 
nusfcer of incentives. 

Personalization is another factor thought to have an influence on 
the rate of mailed questionnaire response. Personalization is defined 
as the process of creating a belief on the part of the respondent that 
he/she is receiving the researcher's individual attention. Dillnen 
(1972), in his esqperimental study, reported an increase of seven 
percentage points (from 85% to 92%) in personalized questionnaires. 
Dillman reviewed the study of C^penter, which reported an increase of 
eight percentage points (frcM 64% to 72%) by using personalized 
procedures, such as individually typed names and prior contact by 
telephoi^. Otter personalized procedures are: individually typed 
letters, handwritten letters, personal salutations, hand applied 
signatures, sxid telefrtione calls to each responctent. The first four 
incentives increase tte rate of response, while the telephone call made 
prior to mailing the questionnaire increases tte velocity of reply 
(Linsky, 1975; Miller and Smith, 1983; Dillman and Frey, 1974, 
Carpenter, 1972; & Dillman, 1983). 

Carpenter (1972), used three treatments to measure tte effect of 
personalization on tte rate of response. In two of them te used non- 
personalization etfects. Hhe results obtained yielded no statistical 
difference asK^ng tte three treatments* 

Anoth^ perscmalization technique is ttet of explaining to tte 
respondent tte importance of tte questionnaire, and how the respoirJ^^nt 
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was selected for the saii?)le. Linsky (1975), by using this technique^t 
obtained an increase of 12.7 percentage points (froa 29 .8 to 42.5 per 
cent). Linsky further reported that Martin in another stiriy found this 
technique to be counterprodiK±ive. Martin's study revealed a decrease 
in response frc«a 33% to 22%. The fact that Martin's respondents were 
recent graduates fx(M high school, while Linsky* s were members of the 
State Nursing Association may have accounted for the discrepancy in 
their results. 

Findings are inconsistent regarding the effectiveness of many of 
the techniques recoaomended for increasing response rate. Complex 
interactions between a particular technique, the nature of the 
population, and the situation In yhldn It Is used apgoar to prevail. 
For that reason, the current study was designed to assess the relative 
ef fectlver^ss of different types of follov-up techniques In Increasing 
responses In a study involving responses frcM multiple Individuals - In 
this case - college administrators. 

Procedures 

Copies of questionnaires to measure professional develops^nt 
(Carpenter, 1979) and rerfonnarK:::e (Ryan, 1983) were mailed to a randm 
saiatple of 693 Chief Student Affairs Officers (CSAO's) yho were members 
of the National Association of Student Personr^l Administrators. They 
were asked to coc^lete and return the measure of professional 
development, and to forward a copy of the performance measure to tl^ir 
supervisor avid to the student goveriraent president for ccaipletlon and 
return. Stac^ped, self -addressed envelopes were Included for the 
retsirns. 
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Since the analysis involved relationships of professional 
development variables, obtained fzoax the CSAO's, with perfonaance 
variables, obtained froa their supervisor and student government 
president, it vas mcessary to the analysis to have all three responses 
In order to have a coa^lete data set for each subject (CSAO) . 
Therefore, to be considered a response, a full set of three instruaoents 
must have been received. Because of this, the definition of wtet 
constituted a response was mre restrictive than in most studies. A 
greater than typical number of refusals was anticipated because of the 
multiple response situation and the fact that the CSdO*s would be 
subjecting themselves to evaluation by their supervisor and a student, 
making the nature of the study ^lensitive. Subjects who chose not to 
participate were asked to indicate their nonpartlcipation using <^ of 
the enclosed return envelopes. 

Cover letters for the initial mailings were all irdividually typed 
on university letterhead and addressed to the CSAO by nasae. Because of 
limited resources, initial mailings were done in two batches, one of 450 
and the second of 243. Chi Square analysis shoved no differences 
between the two batches in the pattern of c<MJiplete responses to follow 
up (Chi Square = 3.08, 3 df, NS). Therefore, the two batches were 
c(^ined for further analyses. 

Foiar response categories of interest were possible: 1) a complete 
response (all three instris)ents returned), 2) a partial response (one or 
two of the three instruments returr^), 3) a decline to participate (a 
written or telejAioned refusal), and 4) a nonresponse (no instruments 
returned and no e^^ession of intent not to respond). 

Pour weeks after the mailing, 91 ccsi^lete sets of instruments had 
been returr^, for only a 13.1% response rate. There were an additional 
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145 partial responses with at least one questionnaire returned, but with 
at least one questionnaire missing. A tele^^ne follov-up vas used to 
atteB?)t to complete these partial responses, results of which are not a 
part of the current study. There were 113 (17.0%) who had indicated 
inability or unwillingness to participate In the study. Their desire 
was acccx&^K)dated and no further attes^t to elicit a response was oBde. 
Hhere were, therefore, 344 subjects available for the study of the 
effectiveness of the various follow-up approaches. Table 1 smsaarlzes 
the response data. 



!I^le 1 
Respor^ Data 





Number 


Percent 


NuB±»r isalled 


693 




A. Refusals (stated) 


158 


22.8% 


B. Nonresponses 


240 


34.6% 


C. Canpleted prior to follow-up 


91 


13.1% 


D. Partial sets prior to follow-up 


(145) 


(20.9%) 


B. Partial ccsopleted after 
follow-up (38.6% of 145) 


56 


8.1% 


F. Coa4)leted responses to follow-up 


31 


4.5% 


G. Total response (C + E ■{- F) 


178 


il5.7% 



Monrespondees were divided into four eqial groups of 86 each by 
systei&atically assigning then to groi^ froa the laaster list (organized 
in Zip Code order). P'^'' low-up approaches were ther. assigned to the four 
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groups by lot* T!ils procedure avoided possible geograE^ical bias. The 
four laethods used were (1) a postcard requesting that the nonrespondent 
respond with an offer to forward another set of materials If needed^ 
(2) an individually ^^ddressed letter typed on letterhead requesting 
their response with an offer to forward another set of aaterlals if 
needed, (3) a photocopied "Dear Colleague:" letter requesting their 
response with a coB4)lete set of materials enclosed, and (4) telephone 
calls to request their participaticm and offering to send additional 
copies of the qi:^t^onnaires if needed. 
Results 

Table 2 below outlines the results cf the study of follow-up 
procedures. 

Table 2 



CcmpletB and Incos^lete Responses 
after Follow-Up Contacts to Nonrespondees 



Type of Contact 


Co!!:|)lete 
Response 


Incoioplete 
Response 


No 

Response 


TeleE^one 


14 


10 


62 


Letter 


6 


2 


78 


Set o£ BBterials 


8 


9 


69 


Post card 


3 


7 


76 



Ilote. Nu3±)er for each type of c^tact was 86. 



The value of Chi Square calculated for the above data of 16.011 
(6 df, p < .05) indicates that there are differences in the frequencies 
o£ coB^lete responses, partial responses and nonresponses to the 
different types of follow-up procedures. Simre the priaary concern was 
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with coeplete responses the partial response and nonrespoase categories 
vere collapsed and an analysis performed on that data. The value of Chi 
Square of 9 .18 (3 df, p < ,05) Indicates that the different types of 
follov-*jq? approach yield differential respcmse rates, with the telefririorie 
follov-up eliciting the hig^iest responses and the postcard the lowest. 



Table 3 

Tiiae of Receipt of Responses 



When Received 


Kaater 


Percent 


CoB!?)leted prior to follow-up 


91 


51.1% 


Partial con?>leted after follow-up 
(38.5% of 145) 


56 


31,5% 


Coi^leted responses to follow-up 
(9.0% of 344) 


31 


17.4% 


Total response 


1?8 


100.0% 



As shown in Table 3, aljiost half of the total nujaber of coB4)lete 
data sets vere coiqpleted after the follow-up. This strcmgly suggests 
that foL survey research requiring responses froa waltiple individuals, 
a follow-^p is critical. Ihe data also Euggests that greater success 
may be possible iCt soliciting the responses necessary to complete 
partial data sets than in obtaining a coiaplete set of responses from 
those se'^B irau 'irtiid^ no response had been obtained. 

cp^all response rate appears low in coe^arisor. with ca^/ent 
r ttailed surveys. However, in the context of a sailtiple 

X :uation with sensitive content, could be considered quite 
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Results of the studiy su;H?ort tte iE^wrtance of personalization 
ana reduction of response cost effects. The telephone follow-up 
resulted in the higlw.st response rate^ followed by the raailiitg of a 
coo5)lete set of materials* Ihis vas followed by an individually typed 
reminder letter^ with the postcard reminder distinctly inferior. 

Hie telephone call was the most personalized of the aE5)roQches, and 
since an additional set of questionnaires was offered, was only 
marginally lower in rediKrtion of response cost than the coe5)lete set of 
naterials, which ^aas not perscmalized but was the sec<HKl most effective 
approach. Ihe individually typed letter was personalized, but not so 
hi^ly as the telephone contact, and high in response cost since 
questionnaires and return envelopes were not routinely provided and had 
to be requested if meded tc respond* Ihe postcard reminder was not 
personalized and hic^ in respons^e cost. 

It would appear that thie degree of personalization was the most 
salient factor, and that reduction in respc»ise cost \ms also involved. 
However, the current study was not designed to contrast these factors so 
a conclusive statement must await further research. 




11 



13 



Pour MetkDds 



References 

Anderson, J. P., & Berdle, D. R. (1975) • Effects on response rates of 

foroal and infonoal questionnaire foliov-up techniques. Journal of 

Applied Psychology^ 60^ 255-257. 
Andreason, A. R« (1970). Personalizing mil questionnaire 

correspondence. Public Opinion ^aarterly^ 34, 273-277. 
Carpenter, D. S. (1979). Hhe professional developooent of sttdent 

affairs workers: An analysis* Dissertation Abstracts 

Internationale 40, 3645A. (University Microfilms No. 80-00985) 
Clcorpenter, B. H. (1975). Personalizing i^il surveys: A replication ard 

reassessnent. Piablic Opinion Quarterl y, 38^ 614-620. 
DillBBn, D. A., & Prey, J. H. (1974) Contribution of personalization 

to nail questionnaire response as an element of a previoiasly tested 

Eaethod. Journal of Applied Psychology , 59, 296-301. 
Futrell, C. M., & Lan^, C. W., Jr. (1981). Effect on i^il survey return 

rates of including questiwrnaires with follow-up letters. 

Perceptual and Motor Skills, 52, 11-15. 
Goyder, J. (1985). Face-to-face interviews and irailed questionnaires: 

The net difference in resporse rate. Public O pinion Quarterly , 49, 

234-252. 

Harvey, L. (1987). Factors affecting response rate to nailed 

qiH2stionnaires: A ccM^rehenslve review. Journal of the Market 

Research Society, 29, 341-353. 
Hawes, J. M. (1987). ThB effects of personal izaticm, source and offer 

on nsall su3:vey response rate and speed. Akron Business and 

Economic tteview , 18(2), 54-63. 




12 



lA 



Fo\n: Methods 



Ifeberlein^ T., & Batoftgartt^r, R. (1978). Factors affecting response 
rales to ©ailed questionnaires: A quantitative analysis of the 
publisted literature^ Aaaerican Sociological Reviev^ 43, 447-462. 

Heberlein. T, A,, 6 Baisagartner, R, (1981) • Is a questionnaire 
necessary in a second Bailing? Public Opinion Quarterly, 45, 102- 
108. 

Hochstim, J. R.^ & AtherK>sopoulos, D. A, (1970) • Personal follov-up in 

a mail survey: Its contribution and its cost. Public Opinion 

Quarterly^ 34/ 69-81. 
House, J. S., Gerber, W., & McSllchael, A. J. (1977). Increasing nail 

questionnaire response: A controlled replication and extejision. 

Public Opinion Quarterly ^ 41, 95-99. 
Lafarecque, D.P. (1978). A respor^ rate esq^rin^nt using nail 

questionnaires. Journal of Marketing^ 42(4), 82-83. 
Linsky, A. S. (1965). A factorial experiment in inducing responses to 

a Hail questionnaire. Sociology ar>d Social Research ^ 49(2), 183- 

189. 

Linsky, A. S. (1975). Stiimilating responses to mailed qiK^stionraires: 

a review. Public Opinion Quarterly ^ 39, 82-101. 
Martin, W. S., Duncan, W. J., 6 Sawyer, J. C. (1984). interactive 

effects of four response rate inducexoents in sail questionnaires. 

College Student Journal , 18, 143-149. 
Miller, L. B., & Ssith, K. L. (1983). Handling nonrespor^ issues. 

Journal of Extension ^ 21, 45-53. 
PtKrel, D. J., Itelson, H. P., & Wteeler, D. N. (1971). Questionnaire 

Follow-up returns as a function of incentives and responder 

characteristics. Vocational Qaidanoe Quarterly ^ 19, 188-193. 



ERIC 



13 



15 



Pour Methods 



Roscoe, A. M*, Lang, D., & Sheth, J. N. (1975) • Pollov-up laethods, 

questionnaire length, and uarket differences in mail surveys. 

Journal o£ Marketing , 39(2), 20-27. 
Ryan, R. E« It^ relationship o£ leactership behavior and demographic 

characteristics to perfors&nce in chief student affairs officers. 

D issertaticm Abstracts International, 44, 1697A. (lAilversity 

MicrofilBS Ko. 83-24357) 
Shackelton, V. J., & Wild, J. M. (1982). Effect of incentives and 

personal contact on response rate to a mail questiom»aire. 

Psychological Reports , 50, 365-366. 
Snelling, W. R. (1969). The intact of personalized sail questionnaires. 

Journal of Bducational Research^ 63(3), 126-129. 
Tedin, K. L., & Hofstetter, C. R. (1982). «ie effect of cost and 

iisportarce factors on the return rate for single and multiple 

mailings. Public Qpinicm Qtarterlv, 46, 122-128. 
Trice, A. D. (1985). Blesients of personalisation in covering letters 

2ey affect response rates in mail surveys: A further analysis of 

Worthen and Valcarce. Psychological Reports^ 58, 82. 
Von Eteisen, R. D. (1979). Post card reminders versus questionnaires 

and mail survey response rates fro© a professional population. 

Journal of. Busir^ss Research^ 7, 1-7. 
Woodward, J. M., a Md(elvie, S. J. (1985). Effects of topical interest 

and mode of address on response to mail surveys. Psychological 

Reports^ 57, 929-930. 
Worthen, B. R., & Valcarce, R. (1985). Relative effectiveness of 

personalized and form covering letters in initial and follow-u^ 

mail surveys. Psychological tteports , 57, 735-744. 



